Skip to content

fix(runner): report failing lifecycle hooks consistently#838

Merged
Chemaclass merged 1 commit into
mainfrom
fix/836-hook-failure-consistency
Jul 19, 2026
Merged

fix(runner): report failing lifecycle hooks consistently#838
Chemaclass merged 1 commit into
mainfrom
fix/836-hook-failure-consistency

Conversation

@Chemaclass

Copy link
Copy Markdown
Member

🤔 Background

Related #836

A failing set_up_before_script produced three different outcomes depending on the hook's last statement and the bash version — including silently ignored failures, off-by-one totals with no message, and a strict test file aborting the whole run mid-suite.

💡 Changes

  • Capture the hook's real exit status (not just the ERR-trap global) and guard the trap's return by frame, so every hook failure is attributed, counted correctly, and the suite continues
  • Reset the runner loop's shell mode after sourcing each file — a file's top-level set -euo pipefail no longer leaks into the runner
  • Run the function cleanup on the hook-failure path and count casualties from the file's own function list
  • Harden two src helpers under strict mode and fix 13 latently strict-broken tests the now-deterministic --strict exposed

https://claude.ai/code/session_01JSeB8UzLCpqst55hAK6dED

A failing set_up_before_script produced three different outcomes
depending on incidental details: attributed failure (plain failing
command), silently ignored (failing `cmd && var=x` guard on Bash 3.2 —
execute_file_hook read only the ERR-trap global and discarded the
hook's real exit status), or silent failures with an off-by-one count
(Bash >= 4, where the ERR trap re-fired in the executor's own scope
and returned before record_file_hook_failure ran).

- capture the hook's real status from $? and fall back to the trap
  global; guard the trap's return by FUNCNAME so the executor-scope
  re-fire records instead of silently bailing (both hook executors)
- reset the runner loop to set +euo after sourcing each file: a test
  file's top-level `set -euo pipefail` leaked into the loop and let a
  failing hook abort the whole run mid-suite with no summary
- count hook-failure casualties from the file's own function list and
  run the #829 function cleanup on the failure path too — previously
  leftover functions from earlier files inflated the totals
- harden two src helpers that broke callers under strict mode:
  is_diff_enabled read an unset var, cleanup_testcase_temp_files
  returned 1 on its skip path
- make 13 latently strict-broken tests capture nested failures with
  `|| code=$?` — the loop reset makes --strict honest and deterministic,
  which exposed them (they already failed standalone on main)
- report_json: a jq-less box now skips instead of failing the hook

Closes #836

Claude-Session: https://claude.ai/code/session_01JSeB8UzLCpqst55hAK6dED
@Chemaclass Chemaclass added the bug Something isn't working label Jul 19, 2026
@Chemaclass Chemaclass self-assigned this Jul 19, 2026
@Chemaclass
Chemaclass merged commit 363cede into main Jul 19, 2026
37 checks passed
@Chemaclass
Chemaclass deleted the fix/836-hook-failure-consistency branch July 19, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant